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IMAGE SYNTHESIS APPARATUS 
BACKGROUND OF THE INVENTION 

1- FIELD OF THE INVENTION: 

The present invention relates to an image 
5 synthesis apparatus usable for observing a state of a 
surface of a subject having a temperature difference, for 
example, usable for diagnosing an inflammation site of 
a human body, and allowing a user to rapidly and accurately 
diagnose a site for diagnosis. 

10 

2. DESCRIPTION OF THE RELATED ART: 

Figure 11 shows a structure of a conventional 
image synthesis apparatus 600 for generating a visible 
stereo image. The image synthesis apparatus 600 includes 

15. a left visible light camera 601 provided for imaging a 
subject from the left of the subject, a right visible light 
camera 602 provided for imaging the subject from the right 
of the subject, and a visible 3D (three-dimensional) image 
synthesis device 603. Conventionally, a visible stereo 

20 image is generated by synthesizing Images obtained by the 
left visible light camera 601 and the right visible light 
camera 602 by the visible 3D image synthesis device 603. 



An infrared camera is also conventionally used for 
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diagnosing a site of a human body or other purposes. An 
infrared camera is used independently, and a physician 
diagnoses the site based on an image obtained by the 
infrared camera. 

5 

The use of a single infrared camera has a problem 
in that the obtained image is not as realistic as an actual 
visual observation , and thus sufficient information 
required for diagnosis is not obtained. Accordingly, the 
10 diagnosis of a subtle inflammation symptom relies heavily 
on the experience and intuition of the physician. 

Japanese Laid-Open Publication No. 9-220203 
discloses a diagnostic apparatus for synthesizing 
15 thermograph image information of a site for diagnosis and 
moire topograph information of the sits. However,, such 
a diagnostic apparatus merely provides a planar thermal 
image and cannot be used to rapidly and accurately grasp 
the state of the site for diagnosis. 

20 

SUMMARY OF THE INVENTION 

According to one aspect of the invention, an image 
synthesis apparatus inoludes a right infrared camera and 
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a left infrared camera; a right visible light camera and 
a left visible light camera; and a first image synthesis 
processing deviae for synthesizing data output from the 
right infrared camera and from the left infrared camera 
5 and data output from the right visible light aamera and 
from the left visible light camera so that a three- 
dimensional thermal image and a three-dimensional visible 
light image are observed by an observer as overlapping 
each other. 

10 

In one embodiment of the invention, the first 
image synthesis processing device includes a synchronous 
signal generator for generating a synchronous signal; a 
seaond image synthesis processing deviae for synthesizing 

15 at least a portion of right infrared image data output 
from the right infrared camera and at least a portion of 
right visible light image data output from the right 
visible light camera in response to a synchronous signal 
generated by the synchronous signal generator so as to 

20 generate right synthesis image data; a third image 
synthesis processing deviae for synthesizing at least a 
'portion of left infrared image data output from the left 
infrared camera and at least a portion of left visible 
light image data output from the left visible light camera 
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in response to a synchronous signal generated by the 
synchronous signal generator so as to generate left 
synthesis image data,- and a data output device for 
outputting the right synthesis image data and the left 
synthesis image data in a prescribed order in response 
to a synchronous signal generated by the synchronous 
signal generator. 

In one embodiment of the invention, the second 
image synthesis processing device synthesizes a portion 
of the right infrared image data and the entire right 
visible light image data. The third image synthesis 
processing device synthesizes a portion of the left 
infrared image data and the entire left visible light 
image data. 

In one embodiment of the invention, the image 
synthesis apparatus further includes a monitor for 
displaying a right synthesis image and a left synthesis 
image in a prescribed order based on the right synthesis 
image data and the left synthesis image data which are 
output from the data output device. 

In one embodiment of the invention, the image 
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synthesis apparatus further includes a polarizer for 
polarizing the right synthesis image in a first direction 
and polarizing the left synthesis image in a seaond 
direction different from the first direction, in response 
5 to a synchronous signal generated by the synchronous 
signal generator. 

In one embodiment of the invention, the . first 
image synthesis processing device includes a synchronous 

10 signal generator for generating a synchronous signal r and 
a data output device for outputting right infrared image 
data output from the right infrared camera, right visible 
light image data output from the right visible light 
camera, left infrared image data output from the left 

15 infrared camera, and left visible light image data. output 
from the left visible light camera in a prescribed order, 
in response to a synchronous signal generated by the 
synchronous signal generator. 

20 In one embodiment of the invention, the first 

image synthesis processing device includes a second image 
synthesis processing device for synthesizing right 
infrared image data output from the right infrared camera 
and left infrared image data output from the left infrared 
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camera so as to generate three-dimensional thermal Image 
data? a third, image synthesis processing device for 
synthesizing right visible light image data output from 
the right visible light camera and left visible light 
image data output from the left visible light camera so 
as to generate three-dimensional visible light image 
data; and a fourth image synthesis processing device for 
synthesizing the three-dimensional thermal image data and 
the three-dimensional visible light image data so as to 
generate three-dimensional overall image data- 
in one embodiment of the invention, the three- 
dimensional thermal image data Includes a plurality of 
temperature levels, and a plurality of color tones are 
respectively assigned to the plurality of temperature 
levels . 

In one embodiment of the invention, the three- 
dimensional overall image data includes three- 
dimensional coordinate data, and the three-dimensional 
overall image data is converted into two-dimensional data 
by transforming the three-dimensional coordinate data 
into two-dimensional coordinate data projected onto a 
prescribed plane in a three-dimensional coordinate space. 
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According to another aspect o£ the invention, an 
image synthesis apparatus includes a slit device 
including a plurality of slits; an infrared directing 
5 device for directing infrared toward a subject through 
the slit device; a plurality of infrared cameras provided 
in a direction substantially perpendicular to a direction 
in which the Infrared is directed toward the subject; and 
an image synthesis processing device for synthesizing a 
10 plurality of thermal image data output from the plurality 
of infrared cameras so as to generate three-dimensional 
thermal image data. 

in one embodiment of the Invention, the image 
15 synthesis processing device generates the three- 
dimensional thermal image data based on a difference in 
shape between heat ray patterns reflected by the subject 
and respectively imaged by the plurality of infrared 
cameras . 

20 

Thus, the invention described herein maJces 
'possible the advantages of providing an image synthesis 
apparatus for allowing a user to rapidly and accurately 
grasp a state of a surface of a subject having a temperature 
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difference and thus allowing a user to rapidly and 
accurately diagnose , for example, an inflammation site 
of a human body. 

' 5 These and other advantages of the present 

invention will become apparent to those skilled in the 
art upon reading and understanding the following detailed 
description With reference to the accompanying figures . 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a schematic diagram illustrating a 
structure of an image synthesis apparatus according to 
a first example of the present invention; 

15 

Figure 2 is a diagram illustrating processing 
performed by an infrared 3D' image synthesis processing 
device in the image synthesis apparatus shown in Figure 
1; 

20 

Figure 3 is a flowchart illustrating an exemplary 
'operation performed by the image synthesis apparatus 
shown in Figure 1; 
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Figure 4 Is a schematic diagram illustrating a 
structure of an image synthesis apparatus according to 
a second example of the present invention; 

5 Figures 5A through 5D, 6, 7A and 7B are eaoh a 

schematic view illustrating a data structure of data 
generated in the image synthesis apparatus shown in Figure 
2; 

10 Figure 8 is a schematic diagram illustrating a 

structure of an image synthesis apparatus according to 
a third example of the present invention; 

Figure 9 is a schematic diagram illustrating a 
15 structure of an image synthesis apparatus according to 
a fourth example of the present invention; 

Figures 10A and 10B show heat ray patterns 
obtained in the image synthesis apparatus shown in Figure 
20 9; and 



Figure 11 is a block diagram illustrating a 
conventional image synthesis apparatus. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 



Hereinafter , the present invention will be 
described by way of illustrative examples with reference 
5 to the accompanying drawings. Identical elements bear 
identical reference numerals in all the figures,, and the 
descriptions thereof will be omitted. 

(Example 1) 

0 Figure 1 shows a structure of an image synthesis 

apparatus loo according to a first example of the present 
invention . The image synthesis apparatus 100 is used for , 
for example, diagnosing an inflammation site of a human 
body as a subject of imaging. 

5 

The image synthesis apparatus 100 includes a left 
Infrared camera 1 , a right infrared camera 2 , a left 
visible light camera 3, a right visible light camera 4, 
and an image synthesis processing device 101. The image 
0 synthesis processing device 101 includes an infrared 3D 
image synthesis processing device 5, a visible light 3D 
image synthesis processing device 6. and an overall 3D 
image synthesis processing device 7 . 
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The left infrared camera 1 and the right infrared 
camera 2 respectively image a subject (not shown) from 
a left position, corresponding to a left eye and a right 
position corresponding to a right eye, and thus generate 
5 Infrared image data. The left visible light camera 3 and 
the right visible light camera 4 respectively image the 
subject from a left position corresponding to a left eye 
and a right position corresponding to a right eye, and 
thus generate visible light image data. 

10 

As shown in Figure 1, left infrared image data 1' 
from the left infrared camera 1 and right infrared image 
data 2' from the right infrared camera 2 are input to the 
infrared 3D image synthesis processing device 5. Left 
15 visible light image data 3» from the left visible light 
camera 3 and right visible light image data 4 f from the 
right visible light camera 4 are input to the visible light 
3D image synthesis processing device 6. 

20 The infrared 3D image synthesis processing device 

5 generates 3D thermal image data 5' for each of the 
'wavelength bands based on the left and right infrared 
image data 1 * and 2 ' . Likewise , the visible light 3D image 
synthesis processing device 6 generates 3D visible light 
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image data 6* for each of the wavelength bands based on 
the left and right visible light image data 3 ' and 4 ' . 
Prom the 3D thermal image data 5 * and the 3D visible light 
image data 6 ' , a 3D thermal image and a 3D visible light 
image are respectively generated as described below. 

The 3D thermal image data 5* and the 3D visible 
light image data 6' are input to the overall 3D image 
synthesis processing device 7. The overall 3D image 
synthesis processing deviae 7 synthesizes the 3D thermal 
image data 5* and the 3D visible light image data 6' , and 
thus generates overall 3D image data 7' . Namely, the 
overall 3D image data 7' includes the 3D thermal image 
data 5" synthesized with the 3D visible light image data 
6 ' . From the overall 3D image data 7 * , an overall 3D image 
is obtained, 

As described ah ove , the image synthesis apparatus 
100 generates an overall 3D image including a 3D visible 
light image synthesized with a 3D thermal image. Such 
an overall 3D image allows the user to grasp both the 
appearance and the thermal distribution m the site for 
diagnosis three -dimensionally . Therefore, the user can 
appreciate the site rapidly and accurately. 
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Specifically, diagnosis of an inflammation site of a human 
"body can be done rapidly and accurately since both an 
external symptom and the temperature distribution of a 
surface of the site are shown. 

S 

A greater number of infrared cameras and visible 
light cameras may be used (for example, four of each) in 
order to obtain a more precise 3D image. 

10 The data 1» and 2* obtained by the left and right 

infrared cameras 1 and 2 are analog data. The data I* 
and 2* are processed by, far example, the infrared 3D image 
synthesis processing device 5. With reference to Figure 
2, an example of processing of data 1' performed by the 

15 infrared 3D image synthesis processing device 5 will be 
described. The data 2* is processed in substantially the 
same manner. 

The left infrared image data 1' is input to a low 
20 pass filter 8. The low pass filter 8 removes an 
unnecessary high frequency component from the left 
infrared image data 1 * to generate an analog left thermal 
image signal 8 ' . The analog left thermal image signal 
8' is output to an ADC (analog to digital converter) 9 
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and a synchronous separation section 13. 

The ADC 9 generates a digital left thermal image 
signal 33, from the analog left thermal image signal 8', 
and outputs the digital left thermal Image signal 31 to 
a multiplexer 10 and a microprocessor 11. The 
microprocessor 11 reads a luminance signal level of the 
digital left thermal image signal 31, and specifies a 
digital color signal 32 to be added from a memory 12 in 
accordance with the luminance signal level. 

The memory 12 has digital color signals 32 stored 
therein as a form of a aolor tone table. The digital color 
signals 32 respectively represent a great number of 
prescribed dolor tones. One of the color signals 32 
specified by the microprocessor 11 is output to the 
multiplexer 10. 

The multiplexer lo has a horizontal and vertical 
retrace line period timing sent from a timing generator 
14. The timing generator 14 generates the horizontal and 
vertical retrace line period timing based on a synchronous 
timing signal sent from the synchronous separation 
section 13. The multiplexer 10 outputs the digital left 
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thermal image signal 31 or the digital color signal 32 
based on the horizontal and vertical retrace line period 
timing* In this manner, the multiplexer 10 outputs a left 
color digital thermal image signal , instead of the digital 
5 color signal 32, in a period which is not the horizontal 
and vertical retrace line period. 



substantially the same manner, and thus a right color 
10 digital thermal image signal is obtained. The 3D thermal 
image data 5* is generated from the left and right color 
digital thermal image signals, whiah is output to the 
overall 3D image synthesis processing device 7 . 

IS The luminance signal levels of the left infrared 

image data 1' and the right infrared image data 2' increase 
as the temperature rises. Utilizing this property, the 
luminance signal level of the analog left thermal image 
signal 8' (and an analog right thermal image signal) is 

20 deteated. From the color tone table stored in the memory 
12, a color tone is selected in accordance with the 
detected luminance signal level. The digital left 
thermal image signal 31 is colored by a digital color 
signal 32 representing the selected color tone. In other 



The right Infrared image data 2' is processed in 
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words, the thermal Image Is colored for each prescribed 
luminance signal level (i.e., for each prescribed 
temperature level data). Alternatively, a plurality of 
colors, for example, red, green, yellow and blue, may be 
assigned to each luminance signal level. 

As described above, the 3D thermal image data 5' , 
generated by the infrared 3D image synthesis processing 
device 5 in correspondence with the 3D visible light image 
data 6 1 , Is colored in accordance with the temperature 
level. Accordingly, the overall 3D image data 7 • obtained 
by synthesizing the 3D thermal image data 5' and the 3D 
visible light image data 6' is also colored in accordance 
With the temperature level. Therefore, the obtained 
overall 3D image allows the user to easily understand the 
temperature distribution of the site for diagnosis 
three-dimensionally . 

The data 7*, which is a 3D image data, can be 
displayed two-dlmensionally . This can be done by freely 
moving the data 7* by coordinate transformation in a 
virtual space by a computer and then projecting the data 
7' onto a plane. 
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Figure 3 is a flowchart illustrating an exemplary 
operation performed by a computer for realizing such a 
two-dimensional display- When the overall 3D image is 
obtained from the overall 3D image data 7 ' , a 3D coordinate 
5 is set in a virtual spaae (world space) (step si). Then, 
the coordinates of 3D image data of the overall 3D image 
are set, and thus configuration data is defined (step S2) . 
When the coordinates of the overall 3D image are set in 
the 3D space, the overall 3D image is processed by rotation 
10 matrix to be coordinate-transformed (step S3). 

Next, the configuration data of the 
coordinate- transformed overall 3D image is converted into 
a 3D polygon patch, (step S4). The overall 3D image la 

15 projected onto an arbitrarily set plane, based on the 
configuration of the 3D polygon patch and the viewpoint 
of the user (step S5), Such processing converts the 
overall 3D image into a two-dimensional coordinate (step 
S6) . The obtained two-dimensional image is displayed on 

20 a screen (step S7). 

As described above, the image synthesis apparatus 
100 according to the first example of the present 
Invention can freely move an overall 3D image, obtained 
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by synthesizing a 3D visible light image and a 3D thermal 
image, in a 3D space and represent an arbitrary 3D image 
as a two-dimensional image. Suah a system allows the user 
to observe the site for diagnosis from a variety of aspects 
5 and thus to perform a more accurate diagnosis. 



7 may count the number of pixels of the overall 3D image 
data 7' which belong to the same temperature range, and 

10 calculate the volume of an arbitrary region in the site 
for diagnosis based on the obtained number. The arbitrary 
region is, for example, an inflammation site. By 
calculating the volume of the inflammation site each time 
the inflammation site is diagnosed, the physician can 

15 obtain an over- time change of the volume of the 
inflammation site (I.e., the degree of swelling) as 
specific numerical data. 

(Example 2} 

20 Figure 4 shows a structure of an image synthesis 

apparatus 200 acaording to a second example of the present 
invention . 



The overall 3D image synthesis processing deviae 



The image synthesis apparatus 200 includes a right 
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infrared camera 201, a right visible light camera 202, 
a left infrared camera 203, a left visible light camera 
204 , an image synthesis processing device 215, and a 
monitor 209. The image synthesis processing device 215 
5 includes a right image synthesis processing device 205, 
a left image synthesis processing device 206, a 
synchronous signal generator 207 , and a data output device 
208. 

1° The right Infrared camera 201 and the left 

Infrared camera 203 respectively image a subject 213 from 
a right position corresponding to a right eye and a left 
position corresponding to a left eye, and thus generate 
right Infrared image data 221 and left Infrared image data 

15 223. The right visible light camera 202 and the left 
visible light camera 204 respectively image the subject 
213 from a right position corresponding to a right eye 
and a left position corresponding to a left eye, and thus 
generate right visible light image data 222 and left 

20 visible light image data 224. 

The right infrared image data 221 and the right 
visible light image data 222 are input to the right image 
synthesis processing device 205. In response to a 
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synchronous signal 217 sent from the synchronous signal 
generator 207, the right image synthesis processing 
device 205 extracts the right infrared image data 221 and 
the right visible light image data 222 which were Imaged 
5 at the same timing. 

The right image synthesis processing device 205 
generates right synthesis image data 225 as shown in 
Figure 5A. The right infrared image data 221 and the right 

10 visible light image data 222 respectively have reference 
points 121 and 122. The right image synthesis processing 
device 205 causes the right infrared image data 221 and 
the right visible light Image data 222 to overlap each 
other, so that the reference points 121 and 122 match each 

15 other. The right image synthesis processing device 205 
can generate the right synthesis image data 225 by, for 
example, arranging pixels of the right infrared image data 
221 and pixels of the right visible light image data 222 
alternately. 

20 

Returning to Figure 4, the left infrared image 
data 223 and the left visible light image data 224 are 
input to the left image synthesis processing device 206. 
In response to a synchronous signal 217 sent from the 
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synchronous signal generator 207 , the left image 
synthesis processing device 206 extracts the left 
infrared image data 223 and the left visible light image 
data 224 -which were imaged at the same timing. 

The left image synthesis processing device 206 
generates left synthesis image data 226 as shown in Figure 
5B. The left infrared image data 223 and the left visible 
light image data 224 respectively have reference points 
123 and 124. The left image synthesis processing device 
206 causes the left infrared image data 223 and the left 
visible light image data 224 to overlap each other, so 
that the reference points 123 and 124 match each other. 
The left image synthesis processing devioe 206 can 
generate the left synthesis image data 226 by, for example, 
arranging pixels of the left infrared image data 223 and 
pixels of the left visible light image data 224 
alternately- 
Returning to Figure 4, the right synthesis image 
data 225 and the left synthesis image data 226 are input 
to the data output device 208. In response to a 
synchronous signal 217 sent from the synchronous signal 
generator 207, the data output device 208 outputs the 
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right synthesis image data 225 and the left synthesis 
image data 226 to the monitor 209 in a prescribed order 
(for example, alternately) as overall synthesis image 
data 227. 

Figure 5C shows a data structure of the overall 
synthesis image data 227 including the right synthesis 
image data 225 and the left synthesis image data 226 
arranged alternately. The overall synthesis image data 
227 includes instruction data for instructing the monitor 
209 to generate a right synthesis image 219 from a 
plurality of right synthesis image data 225 output from 
the output device 208 as shown In Figure 5C, to generate 
a left synthesis image 239 from a plurality of left 
synthesis Image data 226 also output from the output 
device 208 as shown in Figure 5C, and to display the right 
synthesis image 219 and the left synthesis image 239 side 
by side as shown in Figure 4. 

The monitor 209 receives the right synthesis image 
data 225, the left synthesis image data 226 and the 
instruction data, and displays the right synthesis image 
219 and the left synthesis image 239 side by side. A user 
or observer 212 looks at the right synthesis image 219 
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with the right eye and looks at the left synthesis image 
239 with the left eye, and thus perceives an overall 3D 
image including a 3D thermal image and a 3D visible light 
image overlapping each other. 

5 

Figure 5D shows a data structure of alternative 
overall synthesis image data 228. The overall synthesis 
image data 228 may be output to the monitor 209 instead 
of the overall synthesis image data 227. The overall 

10 synthesis image data 228 inoludes the right infrared Image 
data 221, the right visible light image data 222, the left 
infrared image data 223 and the left visible light image 
data 224 arranged in this order repetitiously . in order 
to obtain the overall synthesis image data 228, the right 

15 Infrared image data 221, the right visible light image 
data 222, the left infrared image data 223 and the left 
visible light image data 224 are directly input to the 
data output device 208 without being synthesized by the 
right and left image synthesis processing section 205 and 

20 206. The data output device 208 arranges the data 221 
through 224 as shown in Figure 5D based on a synchronous 
signal 217 sent from the synchronous signal generator 207 . 
The overall synthesis image data 228 is output based on 
a synchronous signal 217, and the monitor 209 displays 
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the overall synthesis image data 228. 

As described above, the image synthesis apparatus 
200 according to the second example of the present 
5 invention provides the right synthesis image data 225 and 
the left synthesis image data 226 to the user 212 ±n the 
form of the overall synthesis image data 227* 
Alternatively , the image synthesis apparatus 200 can 
provide the right infrared image data 221, the right 
10 visible light image data 222, the left infrared image data 
223, and the left visible light image data 224 to the user 
212 in the form of the overall synthesis image data 228, 

In either case, the user 212 sees a 3D thermal 
15 image (obtained by synthesizing the right infrared image 
data 221 and the left infrared image data 223) and a 3D 
visible light image (obtained by synthesizing the right 
visible light image data 222 and the left visible light 
Image data 224) as overlapping each other. As a result, 
20 the user can perceive the two images, i.e. , the 3D thermal 
image and the 3D visible light image, as one synthesized 
image . 



Thus, the image synthesis apparatus 200 in the 



00R0O524 

- 25. - 

second example provides an effect which is equivalent of 
the effect of the image synthesis apparatus 100 shown in 
Figure 1, i.e., the user perceives the synthesized image 
he/she sees as being the same as an overall 3D image 
generated by synthesizing the 3D thermal image data 5" 
and the 3D visible light image data 6 ' by the overall 3D 
image synthesis processing section 7. 

Alternatively, as shown in Figure 6, the right 
image synthesis processing section 205 (Figure 4} may 
synthesize a portion 221* of the right infrared image data 
221 and the entire right visible light Image data 222 to 
generate right synthesis image data 225'. The right 
synthesis Image data 225' can be generated by overlapping 
the portion 221' of the right infrared image data 221 with 
the entire right visible light image data 222 having a 
light luminance or intensity thereof being partially 
lowered. Prom the right synthesis image data 225 * , a 
right synthesis image 229 is obtained. A part of the right 
synthesis image 229 is a thermal image 230 baaed on the 
portion 221*, and the rest of the right synthesis imalge 
229 is a visible light image 231 based on the right visible 
light image data 222 . The left image synthesis processing 
section 206 may perform the same processing. 
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Alternatively, the portion 221" of the right 
Infrared image data 221 and a portion (not shown) of the 
right visible light image data 222 may overlap each other. 

Referring to Figure 4 again , when a physician as 
the user or observer 212 diagnoses a patient as the subject 
231, the physician can predict the position of an 
inflammation site based on the symptoms of the patient 
and the physician's own experience* In such a case, the 
thermal image 230 can be shown for only the predicted 
position and the vicinity thereof while the visible light 
image 231 is shown for the rest. This way, the physician 
can grasp the accurate inflammation site by the visible 
light image 231 and can accurately grasp the fever site 
at the inflammation site by the thermal image 230 at the 
same time. 

The area of the human body In which the thermal 
image 230 is displayed may be an area having a threshold 
temperature which is set by the user or higher, or the 
threshold temperature or lower. Such an area may be 
changed by the intention of the user. 
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As the synchronous signal generator 207, any 
timing circuit is usable. 

Instead of the monitor 209, a pair of right and 
5 left monitors may be used. In this case, the right 
synthesis image data 225 is input to the right monitor, 
and the left synthesis image data 226 is input to the left 
monitor. Alternatively, right synthesis image data 229 
having a data structure shown In Figure 7A may be input 

10 to the right monitor. The right synthesis image data 229 
includes the right infrared image data 221 and the right 
visible light image data 222 arranged alternately. The 
right monitor displays a right infrared image and a right 
visible light image alternately. The alternating 

15 display of the right Infrared image and the right visible 
light image is perceived by the user as being the same 
as the display of the right synthesis image 219 {Figure 
4) . Likewise, left synthesis image data 233 having a data 
structure shown in Figure 7B may be input to the left 

20 monitor. The alternating display of the left infrared 
image and the left visible light image is perceived by 
'the user as being the same as the display of the left right 
synthesis image 239 (Figure 4). 
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Instead of the monitor 209 , a monitor of a head 
mount type (for example, goggles type) Including two 
liquid crystal displays, i.e., for the right eye and the 
other for the left eye may be used. • 

S 

(Example 3) 

Figure 8 shows a structure of an Image synthesis 
apparatus 300 according to a third example of the present 
invention . 

10 

The image synthesis apparatus 300 includes an 
image synthesis processing section 215', a monitor 309, 
and a polarizer 310 in addition to the right infrared 
camera 201, the right visible light camera 202, the left 

15 Infrared camera 203, and the left visible light camera 
204. The image synthesis processing section 215' 
Includes a right image synthesis processing section 205, 
a left image synthesis processing seation 206, a 
synchronous signal generator 207 , and a data output device 

20 218 for outputting overall synthesis image data 227' to 
the monitor 309. The overall synthesis image data 227' 
has the same data structure as that of the overall 
synthesis image data 227 shown in Figure 5C, except that 
the overall synthesis image data 227" includes 
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instruction data for instructing the monitor 309 to 
display the right synthesis image data 225 and the left 
synthesis image data 226 as a right-left synthesized Image 
319. Exaept for the above-mentioned points, the image 
5 synthesis apparatus 300 has substantially the, same 
structure as that of the image synthesis apparatus 200 
shown in Figure 4. 

The monitor 309 receives the right synthesis image 
data 225 and the left synthesis image data 226 which are 
output alternately from the output device 218, as well 
as the instruction data. The monitor 309 alternately 
displays a right synthesis image based on the right 
synthesis image data 225 and a left synthesis image based 
on the left synthesis . image 226, and the observer 212 
perceives the two images alternately output as the 
left-right synthesized image 319. 

The polarizer 310 includes a well known liquid 
20 crystal device or the like. In response to a synchronous 
signal 217 from the synchronous signal generator 207, the 
polarizer 310 polarizes the right synthesis image in a 
first prescribed direction and polarizes the left 
synthesis image in a second prescribed direction whiah 
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is different from the first prescribed direction. After 
the polarization, the polarization directions of the 
right synthesis image and the left synthesis image are- 
different at, for example, 90° from each other. The 
5 observer 212 observes the right synthesis image and the 
left synthesis image, polarized in different directions, 
through polarization glasses 311. The observer 212 sees 
the right synthesis image with his/her right eye and the 
left synthesis image with his/her left eye, and thus 
10 perceives the 3D thermal image and the 3D visible light 
image as one overall 3D image. 

In the third example, the observer 212 is allowed 
to perceive the overall 3D image using the right-left 

15 synthesized image 319, the polarizer 310, and the 
polarization glasses 311. The observer 212 can perceive 
the overall 3D image in other manners . For example , a 
lenticular screen can be used. In this case, the right 
and left synthesis images are each divided into a 

20 plurality of portions along grooves of the lenticular 
screen. The right -left synthesized image 319 is 
perceived as the resultant plurality of portions of the 
right and left synthesis images which are arranged 
alternately. 



00R00S24 

- 31 - 

(Example 4) 

Figure 9 shows a structure of an image synthesis 
apparatus 400 according to a fourth example of the present 
5 invention. 

The image synthesis apparatus 400 includes an 
infrared camera 15 used as a heat source, a slit plate 
16 , infrared cameras 18 and 19 used as sensors, and an 
10 infrared 3D image synthesis processing device 401. 

The infrared cameras 18 and 19 for imaging a 
subject 17 are respectively provided on Y and Z axes, which 
paas through the subject 17 and are perpendicular to each 
15 other. The infrared cameras 18 and 19 are of a passive 
type for sensing infrared and thus generating a thermal 
image. The infrared cameras 18 and 19 provide thermal 
image data of the subject 17. 

20 The alit plate 16 having a plurality of slits 16A 

extended in the direction of the Y axis is provided on 
an X axis which is perpendicular to both the Y and Z axes. 
The infrared camera 15 is provided so as to face the subject 
17 with the slit plate 16 being therebetween. The 
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infrared camera 15 is of an active type for directing 
Infrared toward a subject and imaging the subject based 
on the light reflected by the subject. Infrared emitted 
by the infrared camera 15 is directed to the subject 17 
through the slits 16A of the slit plate 16. Instead of 
the infrared camera 15, any device for directing infrared 
toward a subject is usable. 

With reference to Figures 9, 10A and 10B, Infrared 
emitted by the infrared camera 15 passes through the elite 
16A extended parallel to the Y axis, and are directed to 
the subject 17 as heat rays parallel to the X axle. 

In one example, it is assumed that the subject 17 
has a hemispheric projection as shown in Figures 10A and 
10B. In the case where the subject 17 is imaged by the 
infrared camera 18 in the direction of the Y axis to which 
the slits 16A are parallel, a linear heat ray pattern 510 
as shown in Figure 10B is obtained. In the case where 
the subject 17 is Imaged by the infrared camera 19 in the 
direction of the Z axis perpendicular to the X axis and 
the Y axis, a curved heat ray pattern 511 as shown in Figure 
10A is obtained. The curves of the heat ray pattern 511 
have different curvatures depending on the position of 
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the curved surface of the subject 17. The infrared 
cameras IS and 19 Image infrared emitted by the subject 
17 itself in addition to the patterns 510 and 511- Thus, 
temperature level data representing the temperature 
5 distribution of the surface of the subject 17 is obtained. 

Thermal image data 410 having the heat ray pattern 
510 output by the infrared camera 18, which images the 
subject 17 in the direction of the Y axis, and thermal 
10 image data 411 having the heat ray pattern 511 output by 
the infrared camera 19, which images the subject 17 in 
the direction of the Z axis, are Input to infrared 3D image 
synthesis processing deviae 401. 



15 The infrared 3D Image synthesis processing device 

401 generates 3D thermal image data 412 based on the 
difference in shape between the heat ray patterns 510 and 
511. The 3D thermal image data 412 can be generated by, 
for example, using a well known technology for generating 

20 3D topological data from two-dimensional contours. A 
virtual 3D thermal image 419 obtained from the 3D thermal 
image data 412 1b displayed on a screen 409. The 3D 
thermal image data 412 is colored for each temperature 
level data, and thus the virtual 3D thermal image 419 shows 




0OR00524 

- 34 - 

a temperature distribution of the surface of the subject 
17. 

The image synthesis apparatus 400 in the fourth 
5 example provides the virtual 3D thermal image 419 of the 
subject 17 based on the difference in shape between the 
heat ray patterns 510 and 511. Accordingly, a 3D Image 
of the subject 17 can be provided without requiring 
visible light image data output by a visible light camera. 

10 

since the virtual 3D thermal image 419 is obtained 
based on the thermal image data 410 and 411 having data 
on the shape and the temperature distribution of the 
subject 17, the site for diagnosis of the subject 17 can 
15 be rapidly and accurately observed. 

The image synthesis apparatus 400 may inolude 
three or more infrared cameras for imaging the subject 
17. In this case, the plurality of infrared cameras are 
20 preferably provided on a plane including the Y axis and 
the Z axis. 

The thermal image data 410 and 411 may be 
synthesized with visible light image data obtained by a 
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pair of visible light cameras (not shown) to generate an 
overall 3D image. 

As described in the first example , the virtual 3D 
5 thermal image 419 may be appropriately moved in a 3D space . 

The infrared 3D image synthesis processing device 
401 may count the number of pixels of the 3D thermal image 
data 412 which belong to the same temperature range and 

10 calculate the volume of an arbitrary region in the subject 
17 based on the obtained number. The arbitrary region 
is, for example , an inflammation site. By calculating 
the volume of the inflammation site each time the 
inflammation site is diagnosed, the physician aan obtain 

15 an over-time change of the volume of the inflammation site 
(i.e. , the degree o£ swelling) as specific numerical data. 

According to the present invention, the subject 
of imaging is not limited to a human body. The present 
20 invention is applicable to imaging of , for example, 
creatureB including animals and plants, machines or other 
structures, and scenery including mountains, oceans and 
cltyscapes . 
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According to the present Invention, overall 3D 
image data is generated toy synthesizing 3D thermal image 
data and 3D visible light image data of a subject. 
Therefore, a 3D visible light image of the subject similar 
to the image obtained by visual observation is displayed 
together with a thermal image. Diagnosis of a site of 
the subjeot can be rapidly and aaourately performed. The 
present invention is specifically suitable to diagnosis 
of an inflammation site of a human body. 

In one embodiment of the invention. Infrared Is 
directed to the subject through slits, and thus thermal 
Image data of the subject can be obtained In a prescribed 
direction. Without a visible light camera, both thermal 
data and data on the external shape of the subject are 
obtained. Therefore, diagnosis of a site of the subject 
can be rapidly and accurately performed. 

Various other modifications will be apparent to 
and can be readily made by those skilled in the art without 
departing from the scope and spirit of this invention. 
Accordingly, it is not intended that the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly 



construed. 



